A Wearable Glasses System Development for Spinal Image Fusion and Surgery by Using Mixed Reality Technology

In this paper, an efficient wearable glasses system is proposed for the application of preoperative spinal surgery. The technology is essential to generate a virtual space with a 3D model of the patient from CT images while marker attached [7]. The 3D model is then used to fit the poses of the phantom during surgery by using the PnP method. The patient’s preoperative 3-D image model is registered by projecting it onto the phantom such that the synthetic 3-D model merges with the real patient image, enabling the surgeon to see-through the patient’s anatomy. Glasses with camera are then used to capture the relative pose between patient’s body and viewing direction of the surgeon by marker detection [4] and localization. Experimental results validate on a phantom, and shown the effectiveness of the proposed method. The foremost objective is to assist the surgeons to see-through the vertebral and surgical planning for a minimally invasive spinal surgery.